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(54) Apparatus for determining the soil degree of printed matter 



(57) An IR image input section (10) inputs an IR 
image of printed matter P1 , using IR light having a near- 
infrared wavelength. An edge emphasizing section (1 1) 
executes edge emphasizing processing on the IR 
image. A fold/wrinWe extracting section (12) extracts 
pixels corresponding to a fold or a wrinkle from the 
edge-emphasized image, counts the number of the 
extracted pixels, measures the average density of the 



extracted pixels obtained when the IR image is input 
thereto, and outputs the number and the average den- 
sity of the extracted pixels as feature quantity data. A 
determining section (13) determines the soil degree of 
on the printed matter P 1 due to a fold or a wrinkle on the 
basis of the feature quantity data. 
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Description 

[0001 ] This invention relates to a soil degree deter- 
mining apparatus for determining wrinkles, folds, etc. of 
a printed area of printed matter. 
[0002] Many conventional apparatuses for deter- 
mining soil degree of printed matter employ a method 
for measuring the density of a printed area or a non- 
printed area of printed matter to thereby detect the soil 
degree of the printed matter. Japanese Patent Applica- 
tion KOKAI Publication No. 60-146388. for example, dis- 
closes a method for dividing printed matter into a printed 
area and a non-printed area, setting, as reference data, 
an integration value of light reflected from the printed 
matter or light transmitted through the printed matter, 
and determining whether or not a soil exists on the mat- 
ter. In this method, a soil such as discoloration, a spot, 
blurring, etc., detected as a block change in the density 
of a local area, is measured as a change in the integra- 
tion value (i.e. sum) of the densities of pixels corre- 
sponding to the non-printed area or the printed area. 
[0003] Further, there is a method for accurately 
determining a fold, a wrinkle, etc. of printed matter as a 
linear area changed in density, instead of determining 
dirt as a block change in the density of local area of 
printed matter. Japanese Patent Application KOKAI 
Publication No. 6-27035, for example, discloses a 
method for measuring a fold and wrinkle of a non- 
printed area. 

[0004] As described above, in the prior art the soil 
degree of printed matter is determined by measuring 
integration values of densities of pixels corresponding to 
the printed and non-printed areas of the printed matter, 
or measuring a fold and wrinkle of the non-printed area 
of the printed matter. However, a method for determin- 
ing the soil degree of printed matter by measuring a fold 
and wrinkle of the "printed area" of the matter is not 
employed in the prior art for the following reason. 
[0005] In general, the density of a soil detected as a 
linear area changed in density . (in the case of a fold, 
wrinkle, etc.) is quite different from the density of a 
sheet of plain paper. The conventional method for 
measuring a fold and wrinkle in a "non-printed area" 
uses this density difference. Specif icaHy, differentiation 
processing is performed to emphasize the change in 
density caused at a fold or a wrinkle, thereby extracting 
pixels corresponding to the fold or the wrinkle by binary 
processing, and calculating the number of the pixels or 
the average density of the pixels. Thus, the soil degree 
is measured. 

[0006] On the other hand, concerning the "printed 
area", there is a case where a pattern having lines of dif- 
ferent widths and/or including pattern components of 
different densities of colors is printed in the printed area, 
or where the entire "printed area" is coated with printed 
ink as in photo-offset printing. In an image obtained in 
the prior art by detecting light reflected from or transmit- 
ted through printed matter, a fold or a wrinkle existing in 



its printed area cannot be discriminated therefrom, 
which means that a soil cannot be extracted from the 
printed area. This is because the density of a soil such 
as a fold or a wrinkle is similar to that of the printed area. 
5 Accordingly, it is very difficult in the prior art to extract 
and measure a fold and/or a wrinkle in the printed area. 

[0007] For example, imagine a case where the inte- 
gration value of densities of pixels corresponding to ink 
and a soil on the entire printed area that includes a fold 

w and/or a wrinkle is measured to detect the soil degree of 
the printed area. In this case, it is difficult to discriminate 
the density of ink from the density of a soil of the fold or 
the wrinkle, and the number of pixels corresponding to 
the fold or the wrinkle is smaller than that of the entire 

is printed area. Moreover, variations exist in the density of 
ink of the printed image. For these reasons, a change in 
density due to the fold or the wrinkle cannot be deter- 
mined from the integration value of pixel densities of the 
printed area. 

20 [0008] As described above, the conventional meth- 
ods cannot measure a fold and/or a wrinkle in a printed 
area of the printed matter. 

[0009] In addition, even if a soil on a printed area or 
a non-printed area of printed matter due to a fold or a 

25 wrinkle can be measured, it is still difficult in the prior art 
to discriminate a fold or a wrinkle from a tear that will 
easily occur in an edge portion of the printed matter. 
This is because in the case of a tear differing from the 
case of a hole or a cutout space, it has a linear area 

30 changed in density as in a fold or a wrinkle, if two tear 
areas are aligned with each other and an image of the 
aligned areas is input 

[0010] It is an object of the invention to provide a 
soil degree determining apparatus that can determine, 

35 as humans do. a fold of a printed area of printed matter, 
unlike the conventional apparatuses. 
[001 1 ] It is another object of the invention to provide 
a soil degree determining apparatus capable of discrim- 
inating between a fold and a tear of printed matter. 

40 which cannot be distinguished in the prior art. 

[0012] The present invention uses a phenomenon, 
appearing when an image of to-be-inspected printed 
matter is input using light of a near-infrared wavelength, 
in which the reflectance or the transmittance of a fold or 

45 a wrinkle of the printed matter is much lower than that of 
a printed area or a non-printed area of the printed mat- 
ter. 

[0013] According to one aspect of the invention, 
there is provide a soil degree determining apparatus for 
so determining soil degree of printed matter, comprising: 

image input means for inputting an IR image of 
printed matter to be subjected to soil degree deter- 
mination, using IR light having a near-infrared 
55 wavelength; image extracting means for extracting 
image data in a particular area including a printed 
area, from the IR image input by the image input 
means; changed -section extracting means for 



2 

<EP : 1 01 1 079A1 _l_> 



3 



EP1 011 079 A1 



4 



extracting, on the basis of the image data in the par- 
ticular area extracted by the image extracting 
means, a non-reversible changed section caused 
when the printed matter is folded, thereby providing 
data concerning the changed section; feature quan- 5 
tity extracting means for extracting a feature quan- 
tity indicative of a degree of non-reversible change 
in the particular area, on the basis of the data con- 
cerning the changed section and provided by the 
changed-section extracting means; and determin- 10 
ing means for estimating the feature quantity 
extracted by the feature quantity extracting means, 
thereby determining a soil degree of the printed 
matter. The image input means has an IR filter for 
filtering wavelength components other than the 15 
near-infrared wavelength. 

[001 4] The input of an image of printed matter using 
light of a near-infrared wavelength enables determina- 
tion of a fold of a printed area of printed matter as 20 
humans do, unlike the conventional apparatuses. 
[0015] Furthermore, the present invention can 
detect, by performing image input using light obliquely 
transmitted through printed matter, a gap formed when 
a tear occurs at an edge portion of the printed matter 2s 
and two portions resulting from the tear displace from 
each other, thereby enabling distinguishing of a tear 
from a fold or a wrinkle, which cannot be realized in the 
prior art. Thus, the present invention can obtain a soil 
degree determination result similar to that obtained by so . 
humans. 

[001 6] This summary of the invention does not nec- 
essarily describe all necessary features so that the 
invention may also be a sub-combination of these 
described features. 35 
[001 7] The invention can be more fully under stood 
from the following detailed description when taken in 
conjunction with the accompanying drawings, in which: 

FIGS. 1A and 1B are views illustrating an example 40 
of printed matter to be checked in a first embodi- 
ment, and an example of an IR image of the printed 
matter; 

FIGS. 2A to 2C are graphs illustrating examples of 
spectral characteristics of a printed area of printed 45 
matter; 

FIGS. 3A and 3B are views useful in explaining the 
relationship between a light source and bright and 
dark portions of printed matter due to a fold of the 
matter when performing reading processing by so 
using reflected light; 

FIG. 4 is a block diagram showing the structure of a 
soil degree determination apparatus, according to 
the first embodiment, for determining a soil on 
printed matter; 55 
FIGS. 5A and 5B are views illustrating an example 
of an arrangement of an optical system that is incor- 
porated in an IR image input section and uses 



transmitted light, and an example of an arrange- 
ment of an optical system that is incorporated in the 
IR image input section and uses reflected light, 
respectively; 

FIG. 6 is an example of an image input timing chart; 
FIGS. 7A and 7B are views showing examples of 
images of printed matter taken into an image mem- 
ory; 

FIGS. 8A and 8B are views illustrating examples of 
vertical and horizontal filters to be used in edge 
emphasizing processing; 

FIG. 9 is a block diagram showing in more detail the 
structure of the soil degree determination appara- 
tus according to the first embodiment; 
FIG. 1 0 is a f fowchart useful in explaining the proce- 
dure of determination processing performed in the 
first embodiment; 

FIGS. 1 1 A and 1 1 B are views illustrating an exam- 
ple of printed matter to be checked in a second 
embodiment, and an example of an IR image of the 
printed matter; 

FIG. 12 is a graph showing examples of spectral 
characteristics in a printed area of printed matter; 
FIG. 13 is a block diagram illustrating the structure 
of a soil degree determination apparatus, according 
to the second embodiment, for determining soil 
degree of printed matter; 

FIG. 1 4 is a flowchart useful in explaining the proce- 
dure of extracting and measuring pixels in line using 
Hough transform; 

FIG. 1 5 is a f fowchart useful in explaining the proce- 
dure of extracting and measuring pixels in line using 
projective processing on an image plane; V 
FIG. 1 6 is a flowchart useful in explaining the proce- 
dure of determination processing performed in the 
second embodiment; 

FIG. 1 7 is a view illustrating an example of printed 
matter to be checked in a third embodiment; 
FIG. 1 8 is a block diagram illustrating the structure 
of a soil degree determination apparatus, according 
to the third embodiment, for determining soil degree 
of printed matter; 

FIGS. 19A to 19D are views useful in explaining 
examples of maximum/minimum filtering operations 
and difference data generation, using one-dimen- 
sional data; 

FIG. 20 is a flowchart useful in explaining the proce- 
dure of determination processing performed in the 
third embodiment; 

FIGS. 21 A to 21C are views showing examples of 
printed matter to be checked in a fourth embodi- 
ment, and its IR image and to-be-masked areas; 
FIG. 22 is a block diagram illustrating the structure 
of a soil degree determination apparatus, according 
to the fourth embodiment, for determining soil 
degree of printed matter; 

FIG. 23 is a flowchart useful in explaining the proce- 
dure of mask area setting processing; 
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FIG. 24 is a flowchart useful in explaining the proce- 
dure of determination processing performed in the 
fourth embodiment; 

FIG. 25 is a view showing an example of printed 
matter to be checked in a fifth embodiment; s 
FIGS. 26A and 26B are views showing examples of 
tears formed in printed matter; 
FIG. 27 is a block diagram illustrating the structure 
of a soil degree determination apparatus, according 
to the fifth embodiment for determining soil degree io 
of printed matter; 

FIGS. 28A and 28B are views illustrating examples 
of arrangements of an optical system, using light 
transmitted through the printed matter, which is 
used in an IR image input section; is 
FIG. 29 is a flowchart useful in explaining the proce- 
dure of determination processing performed in the 
fifth embodiment; 

FIG. 30 is a block diagram illustrating the structure 
of the soil degree determination apparatus in more 20 
detail, according to the fifth embodiment for deter- 
mining soil degree of printed matter; 
FIG. 31 is a view showing a state in which printed 
matter is transferred when inputting an image using 
transmitted light; 25 
FIG. 32 is a block diagram illustrating the structure 
of a soil degree determination apparatus, according 
to a sixth embodiment, for determining soil degree 
of printed matter; 

FIGS. 33A and 33B are schematic top arid perspec- 30 
tive views, respectively, illustrating a printed matter 
transfer system used for the transfer shown in FIG. 
31; and 

FIG. 34 is a flowchart useful in explaining the proce- 
dure of determination processing performed in the 35 
sixth embodiment. 

[0018] The embodiments of the invention will be 
described with reference to the accompanying draw- 
ings. 40 
[001 9] First, soil on printed matter to be determined 
in this invention will be described. In the invention, soil 
on printed matter includes blemishes such as "folds", 
"wrinkles", "tears" and "cutout spaces". The term Told" 
implies, for example, an uneven portion which has 45 
occurred in a printed area when flat printed matter is 
deformed, and which cannot be restored to its original 
state. For example, the fold indicates a linear deformed 
portion which will occur when the printed matter is 
folded about its width-directional center line, and the so 
location of which is substantially known in advance!. 
[0020] On the other hand, "wrinkle" indicates a 
deformed uneven portion which has occurred when the 
printed matter is deformed, and which cannot be 
restored to its original state, as in the case of the fold, ss 
However, in this case, the deformed uneven portion is a 
curved portion or a linear portion occurring when the 
printed matter is bent or rounded. 
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[0021] Tear" indicates a portion of a certain length 
cut from an edge portion of printed matter and having 
no cutout 

[0022] "Cutout space" is formed by cutting and 
removing an edge portion of printed matter. Further, 
"hole" indicates, for example, a circular hole, formed in 
printed matter. 

[0023] Soil includes, as well as the above-men- 
tioned ones, scribbling, the entire stain, yellowish por- 
tions, greasy stains, blurred printing, etc. 
[0024] A first embodiment of the invention will now 
be described. 

[0025] FIG. 1A shows an example of a soil on 
printed matter to be detected in the first embodiment. 
Printed matter P1 shown in FIG. 1 A consists of a printed 
area R1 and a non-printed area Q1 . The printed area 
R1 includes a center line SL1 that divides, into left and 
right equal portions, the printed matter P1 that has a 
longer horizontal side than a vertical side in FIG. 1A. 
Assume that soiling such as a fold or a wrinkle is liable 
to occur along the center line SL1 , and that ink printed 
on the printed area R1 is mainly formed of chromatic 
color ink 

[0026] FIGS. 2A to 2C show examples of spectral 
characteristics of a sheet of paper, chromatic color ink, 
and a fold or a wrinkle. Specifically, FIG. 2A shows the 
tendency of the spectral reflectance of the paper sheet. 
The paper sheet is generally white. FIG. 2B shows the 
tendency of the spectral reflectance of a printed area of 
the paper sheet in which the chromatic color ink is 
printed. It is a matter of course that various colors such 
as red, blue, etc. have different spectral reflectance 
characteristics. The tendency of the spectral reflectance 
characteristics of these chromatic colors is illustrated in 
FIG. 2B. FIG. 2C shows the tendency of the spectral 
reflectance characteristic of a fold or a wrinkle occurred 
in the printed area R1 or the non-printed area Q1, in 
relation to the tendency of the spectral reflectance char- 
acteristics of the paper sheet and the chromatic color 
ink. 

[0027] In general, as is shown in FIG. 2B, the spec- 
tral reflectance characteristic of chromatic color ink 
printed on a paper sheet indicates that the reflectance 
does not significantly vary within a visible wavelength 
range of 400 to 700 run, but substantially increases to 
the reflectance of the paper sheet shown in FIG. 2A in a 
near-infrared wavelength range of 800 nm or more. 
[0028] On the other hand, at a fold or a wrinkle 
which is seen darkly as described later, the reflectance 
does not greatly vary even when the wavelength of light 
varies from the visible wavelength range to the near- 
infrared wavelength range of 800 nm. Although FIGS. 
2A to 2C show the spectral reflectance characteristics 
between the wavelengths of 400 nm and 800 nm, the 
reflectance does not greatly vary in a near-infrared 
wavelength range of 800 nm to 1000 nm, unlike the vis- 
ible wavelength range, but is substantially equal to the 
reflectance obtained in the wavelength range of 800 nm. 
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[0029] As is evident from FIG. 2C, the reflectances 
of the chromatic color ink and the fold or the wrinkle do 
not greatly differ from each other in a visible wavelength 
range of 400 nm to 700 nm, but differ in the near-infra- 
red wavelength rage of 800 nm to 1000 nm. Moreover, 
the reflectances of the paper sheet and the fold or the 
wrinkle greatly differ from each other over the entire 
wavelength range. 

[0030] This means that input of an image obtained 
by radiating the printed matter P1 with light having a 
near-infrared wavelength of 800 nm to 1 000 nm enables 
separation or extraction of a dark portion due to a fold or 
a wrinkle from a paper sheet (Q1) and chromatic color 
ink (R1 ), as is shown in FIG. 2C. 
[0031] A description will then be given of a case 
where image inputting is performed by transmitting, 
through the printed matter P1, the light with the near- 
infrared wavelength of 800 nm to 1000 nm. The "spec- 
tral transmittance" of chromatic color ink does not signif- 
icantly vary within a visible wavelength range of 400 to 
700 nm as in the case of the spectral reflectance shown 
in FIG. 2B, but substantially increases to the transmit- 
tance of the paper sheet in a near-infrared wavelength 
range of 800 nm to 1 000 nm. 

[0032] On the other hand, at a fold or a wrinkle, the 
spectral transmittance is significantly lower than that of 
the paper sheet as in the case of the spectral reflect- 
ance shown in FIG. 2C, since the paper sheet is bent 
and light reflects diffusely from the bent paper sheet. 
Accordingly, the fold or the wrinkle can be extracted 
using transmitted light of a near-infrared wavelength, as 
in the case of using reflected light of a near-infrared 
wavelength when the fold or the wrinkle is seen darkly. 
[0033] A description will now be given of a case 
where a fold or a wrinkle is seen darkly or brightly. 
Where a fold or a wrinkle projects on the opposite side 
of flat printed matter to a light source as shown in FIG. 
3A, a portion indicated by "dark portion" has a lower 
brightness than the other flat areas of the paper sheet 
and hence is seen darkly, since the amount of light from 
the light source is small. 

[0034] Further, a portion indicated by "bright por- 
tion" in FIG. 3A has a higher brightness than the other 
flat areas of the paper sheet and hence is seen brightly, 
since the bent printed surface of the "bright portion" 
reflects light from the light source to a sensor. 
[0035] On the other hand, where a fold or a wrinkle 
projects on the same side of the flat printed matter as 
the light source as shown in FIG. 3B, a portion indicated 
by "bright portion" has a higher brightness for the same 
reason as in the "bright portion" in FIG. 3A and hence is 
seen brightly. Further, a portion indicated by "dark por- 
tion" in FIG. 3B has a lower brightness for the same rea- 
son as in the "dark portion" in FIG. 3A and hence is 
seen darkly. 

[0036] As described above, in the case of using 
reflected light, the brightness of a fold or a wrinkle 
greatly varies depending upon the bending direction or 



angle of the printed matter or upon the angle of radia- 
tion. However, the bright portion of the fold or the wrin- 
kle has a higher brightness than the other flat paper 
sheet areas, and its dark portion has a lower brightness 
5 than them. Using this phenomenon, the accuracy of 
detection of a fold or a wrinkle of a printed area can be 
enhanced. 

[0037] FIG. 4 schematically shows the structure of 
a soil degree determination apparatus, according to the 
10 first embodiment, for determining a soil on printed mat- 
ter, 

[0038] An I R image input section 1 0 receives image 
data corresponding to light with a near-infrared wave- 
length (hereinafter referred to as "I R") of 800 nm to 

is 1000 nm reflected from or transmitted through the 
printed matter P1, and then extracts, from the input 
image data, image data contained in a particular area of 
the printed matter P1 which includes the printed area 
R1. An edge emphasizing section 11 performs edge 

20 emphasizing processing on the image data contained in 
the particular area and extracted by the IR image input 
section 10. 

[0039] A fold/wrinkle extracting section 12 binarizes 
the image data obtained by the edge emphasizing 

25 processing in the edge emphasizing section 1 1 , thereby 
extracting pixels having greatly different brightnesses 
and performing feature quantity extraction processing 
on the pixels. A determining section 13 determines the 
soil degree of the printed matter P1 on the basis of each 

30 feature quantity extracted by the fold/wrinkle extracting 
section 12. . . 

[0040] The operation of each of the above-men- 
tioned sections will be described in detail. 
[0041] The IR image input section 10 detects the 

35 printed matter P1 transferred, using a position sensor, 
and reads, after a predetermined time, IR optical infor- 
mation concerning the printed matter P1 with the 
printed area R1 , using a CCD image sensor. The IR 
image read by the image sensor is subjected to A/D 

40 conversion and stored as digital image data in an image 
memory. The particular area including the printed area 
R1 is extracted from the stored image data. After that, 
the other processes including the process by the edge 
emphasizing section 1 1 are executed. 

45 [0042] FIGS. 5A and 5B illustrate an arrangement 
of an optical system that is incorporated in the IR image 
input section 10 and uses transmitted light, and an 
arrangement of an optical system that is incorporated in 
the IR image input section 10 and uses reflected light, 

so respectively. In the case of the optical system using 
transmitted light, a position sensor 1 is provided across 
the transfer path of the printed matter P1 as shown in 
FIG. 5A. A light source 2 is located downstream of the 
position sensor 1 with respect to the transfer path and 

55 below the transfer path with a predetermined space 
defined therebetween. 

[0043] The light source 2 is a source of light includ- 
ing IR light. Light emitted from the source 2 is transmit- 
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ted through the printed matter P1. The transmitted light 
passes through an IR filter 3 located on the opposite 
side to the light source 2 with respect to the printed mat- 
ter P1, thereby f Qtering light, other than the IR light con- 
tained in the transmitted light The IR light is converged 5 
onto the light receiving surface of a CCD image sensor 
5 through a lens 4. 

[0044] The CCD image sensor 5 consists of a one- 
dimensional line sensor or of a two-dimensional sensor. 
When the sensor 5 consists of the one-dimensional line 10 
sensor, it is located in a direction perpendicular to the 
transfer direction of the printed matter. 
[0045] On the other hand, in the case of the optical 
system using reflected light the optical system differs, 
only in the position of the light source 2, from the optical 1 s 
system using transmitted light shown in FIG. 5A. Specif- 
ically, in this case, the light source 2 is located on the 
same side as the IR filter 3, the lens 4 and the CCD 
image sensor 5 with respect to the transfer path, as is 
shown in FIG. 5B. . 20 
[0046] In this case, light is obliquely applied from 
the light source 2 to the transfer path, and light reflected 
from the printed matter P1 is converged onto the light 
receiving surface of the CCD image sensor 5 via the IR 
filter 3 and the lens 4. 25 
[0047] Referring then to FIG. 6, the timing of image 
input will be described. When the printed matter P1 
passes through the position sensor 1 , the position sen- 
sor 1 detects that light is shaded by the printed matter 
P1 . At the detection point in time, a transfer clock signal 30 
starts to be counted. In the case where the CCD image 
sensor 5 consists of a one-dimensional line sensor, a 
one-dimensional line sensor transfer-directional effec- 
tive period signal changes from ineffective to effective 
after a first delay period, at the end of which the count 35 
value of the transfer clock signal reaches a predeter- 
mined value. This signal keeps effective for a longer 
period than the shading period of the printed matter P1 , 
and then becomes ineffective. 

[0048] Image data that includes the entire printed 40 
matter P1 is obtained by setting the period of the one- 
dimensional line sensor transfer-directional effective 
period signal longer than the shading period of the 
printed matter P1. The first delay period is set in 
advance on the basis of the distance between the posi- 45 
tion sensor 1 and the reading position of the one-dimen- 
sional line sensor, and also on the basis of the transfer 
rate. 

[0049] Further, in the case where the CCD sensor 5 
consists of a two-dimensional sensor, the shutter effec- so 
tive period of the two-dimensional sensor is set effective 
for a predetermined period after a second delay period, 
at the end of which the count value of the transfer clock 
signal reaches a predetermined value, thereby causing 
the two-dimensional sensor to execute image pick-up ss 
within the shutter effective period. 
[0050] Like the first delay period, the second delay 
period is set in advance. Further, although in this case, 



the two-dimensional sensor picks up an image of the 
transferred printed matter PI while the shutter effective 
period of the sensor is controlled, the invention is not 
limited to this, but the two-dimensional sensor can be 
made to pick up an image of the transferred printed mat- 
ter P1 while the emission period in time of the light 
source is controlled. 

[0051 ] FIGS. 7A and 7B illustrate examples where a 
particular area including the printed area R1 is 
extracted from input images. The hatched background 
has a constant density, i.e. has no variations in density. 
Irrespective of whether the printed matter P1 does not 
incline as shown in FIG. 7A, or it inclines as shown in 
FIG. 7B, respective areas are extracted, in which the 
density varies by a certain value or more over a con- 
stant distance toward the opposite sides from the width- 
directional central position of an input image of the 
printed matter P1. 

[0052] The edge emphasizing section 11 will be 
described. The edge emphasizing section 1 1 performs 
a weighting operation on (3 x 3) pixels adjacent to and 
including a target pixel (a central pixel) as shown in FIG. 
8A, thereby creating a vertical-edge-emphasized 
image. Specifically, eight values obtained by adding 
weights shown in FIG. 8A to the densities of the adja- 
cent pixels are further added to the density of the target 
pixel, thereby changing the density of the target pixel. 
The edge emphasizing section 1 1 further obtains a hor- 
izontal-edge-emphasized image by executing a weight- 
ing operation on the (3 x 3) pixels adjacent to and 
including the target pixel as shown in FIG. 8B. By the 
vertical- and horizontal-edge-emphasizing process, a 
change in density at a fold or a wrinkle is emphasized in 
an image input using reflected or transmitted light In 
other words, a change in density from a bright portion to 
a dark portion or vice versa at a fold shown in FIG. 3A 
or 3 B is emphasized. 

[0053] The fold/Wrinkle extracting section 12 will be 
described. In this section, the vertical- and horizontal- 
edge-emphasized images obtained by the edge empha- 
sizing section 11 are subjected to binary processing 
using an appropriate threshold value, thereby vertically 
and horizontally extracting high-value pixels which typi- 
cally appear at a fold or a wrinkle. 
[0054] After that, the number of extracted pixels, 
and the average density of the extracted pixels (i.e. the 
average density of an original image), which is assumed 
when the original image is input to the IR image input 
section 10, are obtained vertically and horizontally. 
Moreover, concerning the pixels extracted by binariza- 
tion after the vertical-edge-emphasizing processing, 
variance from horizontal average position is obtained. 
More specifically, the variance is obtained using the fol- 
lowing equation (1), in which a number (n + 1) of 

extracted pixels are represented by (ik, jk) [k ■ 0. 1 

n]: 
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[0055] Each of the thus-obtained feature quantities s 
is output to the determining section 13. 
[0055] The determining section 13 win now be 
described. The determining section 13 determines the 
soil degree of the printed matter P1 on the basis of each 
feature quantity data Hern extracted by the toW/wrinWe 10 
extracting section 12. A reference value used in this 
determination wfll be described later. 
[0057] Referring to FIG. 9, the structure of the soil 
degree determination apparatus according to the first 
embodiment will be descrtoed in detail. FIG. 9 is a block is 
diagram showing the structure of the soil degree deter- 
mination apparatus. 

[0058] As is shown in the figure, a CPU (Central 
Processing Unit) 31, a memory 32, a display section 33, 
an image memory control section 34 and an image-data 20 
l/F circuit 35 are connected to a bus 36. 
[0059] |R image data corresponding to the printed 
matter PI input by the IR image input section 10 is Irput 
to the image memory control section 34 on the basis of 
a detection signal from the position sensor 1 at a point 25 
in time controlled by a timing control circuit 37. The 
operations of the IR image input section 10, the position 
sensor 1 and the timing control circuit 37 have already 
been described with reference to FIGS. 5 and 6. 
[0060] IR image date input to the image memory so 
control section 34 is converted into cfigrtaJ image data by 
an A/D conversion circuit 38, and stored in an image 
memory 40 at a point in time controlled by a control cir- 
cuit 39. The image data stored in the image memory 40 
is subjected to image processing and determination as 
processing performed under the control of the CPU 31 
in accordance with programs corresponding to the edge 
emphasizing section 11, the fold/wrinkle extracting sec- 
tion 12 and the determining section 13 shown in FIG. 4. 
The memory 32 stores these pr ograms . The display 40 
section 33 cfisplays the determination results of the CPU 
31. 

[0061] The image data stored in the image memory 
40 can be transferred to an external device via the bus 
36 and the image-data l/F circuit 35. The external 4S 
device stores, in an image storage device such as a 
hard disk, transferred image data on a plurality of pieces 
of printed matter P1. Further, the external device calcu- 
lates, on the basis of the image data on the plurality of 
the printed matter pieces, a reference value tor soil so 
degree determination which will be described later. 
[0062] Referring then to the flowchart of FIG. 10, 
the entire procedure of the determination processing 
performed in the first embodiment will be described. 
[0063] First, IR image of the printed matter P1 is ss 
input using the IR image input section 10 (S1). and a 
particular area including the printed area R1 is 
extracted from the input image (S2). Subsequently, the 



edge emphasizing section 1 1 performs vertical and hor- 
izontal edge emphasizing processing, thereby creating 
respective edge emphasized images (S3, S4). 
[0064] After that the foU/wrinWe extracting section 
12 performs binarization processing on each of the ver- 
tical and horizontal edge emphasized images, using an 
appropriate threshold value, thereby creating binary 
images (S5. S6). The number of vertical edge pixels 
obtained by the binarization processing Is counted (S7). 
and the average density of the extracted pixels, which is 
obtained when the original image is input thereto, is cal- 
culated (S8). thereby calculating variance of horizontal 
positions (or coordinate values) (S9). Similarly, the 
number of horizontally extracted pixels is counted 
(S10). and the average density of the extracted pixels, 
which is obtained when the original image is input 
thereto, is calculated (S1 1). 

[0065] Then, the determining section 13 deter- 
mines the soil degree on the basis of each calculated 
feature quantity data item (the number of extracted pix- 
els, the average density of the extracted pixels, the var- 
iance) (Si 2), and outputs the soil degree determination 
result (S13). 

[0066] A description will now be given of the crea- 
tion of the reference value used for the determining sec-, 
tion 13 to determine the soa degree based on each 
feature quantity data item. First, image data on the 
printed matter P1 is accumulated in an external image 
data accumulation device via the image data l/F circuit 
35. The inspection expert estimates the accumulated 
image samples of the printed matter P1 to thereby 
arrange the image samples in order from "clean" to 
"dirty". 

[0067] Furthermore, each image data (master data) 
item accumulated in the image data accumulation 
device is once subjected to each feature quantity data 
extraction processing performed at the steps S2 - S1 1 
in FIG. 10 by a general operation processing device. As 
a result a plurality of feature quantities are calculated 
for each sample of printed matter. After that, a combina- 
tion rule used in combination processing for containing 
the feature quantities is learned or determined so that 
the soil degree of each piece of printed matter deter- 
mined by the combination processing of the feature 
quantities win become closer to the estimation result of 
the expert 

[0068] A method for obtaining the soil degree by lin- 
ear combination is considered as one of methods tor 
obtaining the combination rule by learning. For exam- 
ple, a total estimation Y indicative of how degree each 
piece of printed matter is soiled is determined using 
weight data aO, a1 an (the aforementioned refer- 
ence value) as in the following linear combination for- 
mula (2). supposing that the number of extracted 
feature quantity data items on each printed matter piece 
is (n + 1), and that the feature quantities are repre- 
sented by f1.f2 fn: 



7 



BNSDOCID: <EP 



1011079A1_L> 



BNS page 7 



13 



EP1 011 079 A1 



14 



Y - a0 + a1 xfl+a2xt2 + • • • + 

x fn • • • 



(2) 



[0069] A second embodiment of the invention wffl s 
now be described. 

[0070] In the above-described first embocfiment 
chromatic color ink is printed in the printed area R1 of 
the printed matter P1. if. however, ink which contains 
carbon is used as wefl as the chromatic color ink, a fold 10 
or a wrinkle cannot be extrac t ed by the binarization 
processing performed in the foW/wrinWe extracting sec- 
tion 12 in the first embocfiment 
[0071] FIG. 11A shows an example of a serf on 
printed matter/which cannot be extracted in the first is 
embodiment Printed matter P2 shown in FIG. 11 A con- 
sists of a printed area R2 and a non-printed area Q2. 
The printed area R2 includes a center One SL2 that 
divides a printed pattern and the printed matter P2 into 
two portions in the horizontal direction. Assume that 20 
soffing such as a fold or a wrinkle is Dabie to occur near 
the center One SL2. as in the case of the printed matter 
P1 having the center One SL1. 
[0072] The ink printed on the printed area R2 con- 
tains, for example, black ink containing carbon, as wefl 25 
as chromatic color ink. FIG. 12 shows examples of 
spectral characteristics of Hack ink containing carbon, 
and a mixture of black ink and chromatic color ink. 
[0073] In the case of the chromatic color ink, its 
reflectance greatly differs between a visWe wavelength 30 
range of 400 ran to 700 ran and a near-infrared wave- 
length range of 800 ran to 1000 ran, and abruptly 
increases when the wavelength exceeds about 700 ran 
In the case of using a mixture of chromatic color ink and 
black ink containing carbon, its reflectance is lower than 35 
that of the chromatic color ink itself in the near-infrared 
wavelength range of 800 ran to 1000 ran. In the case of 
using black ink containing carbon, its reflectance little 
varies between the visUe wavelength range of 400 ran 
to 700 ran and the near-infrared wavelength range of 40 
800 ran to 1000 ran. 

[0074] K a fold or a wrinkle is attempted to be 
extracted from the printed matter P2 having the above- 
described printed area R2 by the same method as 
errployed in the first embocfiment noise win be 45 
extracted from a portion of the printed area R2, which 
contains ink other than the chromatic color ink, as is 
shown in FIG. 11B. Because of pixels detected as 
noise, the foW/wrinWe extraction processing executed in 
the first embodiment cannot be employed. so 
[0075] However, it should be noted that high-value 
pixels, which typically appear at a fold, are arranged in 
line. Using this feature enables the detection of a 
straight line from a binary image in which the ink-printed 
portion is detected as noise, thereby extracting a fold. In ss 
the second embodiment described below, the sofl 
degree of the printed matter P2, which cannot be deter- 
mined in the first embocfiment can be determined. 



[0076] FIG. 13 is a schematic block cfiagram illus- 
trating the structure of a soil degree deter min ation 
apparatus, according to the second embodiment for 
determining sofl degree of printed matter. The soil 
degree determination apparatus of the second embodi- 
ment differs from that of the first embocfiment in the fol- 
lowing points: The edge emphasizing section 11 in the 
first embodiment creates horizontal and vertical edge 
emphasized images, whereas the corresponding sec- 
tion 1 1 in the second embocfiment creates only a verti- 
cal edge emphasized image. Further, in the second 
embocfiment the foWwrinkle extracting section 12 
employed in the first embodiment is replaced with an 
edge voting section 14 and a B near-fine extracting sec- 
tion 1S 

[0077] The edge voting section 14 and the D near- 
line extracting section 15 wiD be descrfoed. There are 
two processing metho ds that should be changed 
depending upon spaces to be voted. First a description 
will be given of the case of using Hough transform. 
[0078] In the edge voting section 14. the vertical 
edge emphasized image obtained in the edge empha- 
sizing section 11 is subjected to binarization using an 
appropriate threshold value; thereby extracting high- 
value pixels which typically appear at a told or a wrinkle. 
At this time, the ink-printed portion is extracted together 
with noise. 

[0079] The flowchart of FIG. 14 illustrates the pro- 
cedure of processing executed in the edge voting sec- 
tion 14 and the linear-fine extracting section 15. The 
edge voting section 14 performs Hough transform as 
known proce ssin g on the obtained binary image, 
thereby voting or plotting the extracted pixels including 
noise on a Hough plane using "distance p* and "angle 6" 
as parameters (S21). Supposing that a number n of 
extracted pixels including noise are represented by (xk. 
yk)[k ■ 1, .... n], each pixel is voted on the Hough plane 
on the basis of the following equation (3): 



p -xkxCOSe+ykx SIN6 



(3) 



[0080] The parameters p and e. which serve as the 
axes of the Hough plane, are divided into equal units, 
and accordingly, the Hough plane (p. e) is divided into 
squares with a certain side length. Where one pixel is 
subjected to Hough transform, a curve is formed on the 
Hough plana One vote is voted in any square through 
which the curve passes, and the number of votes is 
counted in each square Where a square having maxi- 
mum votes is obtained, one linear line is determined 
using the equation (3). 

[0081] The BneaHine extracting section 15 exe- 
cutes the following processing. First the counted value 
of votes in each square on the Hough plane (p, e) is sub- 
jected to binarization using an appropriate threshold 
value, thereby extracting a linear-line parameter (or lin- 
ear-line parameters) indicating a linear line (or linear 
Ones) (S22). Subsequently, pixels, which are included in 
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the pixels constituting a linear line in the printed area 
determined by the extracted linear-line parameters), 
and which are already extracted by the binarization, are 
extracted as pixels corresponding to a fold (S23). After 
that, the number of pixels on the extracted linear line is 5 
counted (S24), thereby measuring the average density 
of the extracted pixels, which is obtained when the orig- 
inal image is input thereto (S25). 

[0082] As described above, extraction of pixels 
located only on the detected linear line can minimize the 10 
influence of background noise, resulting in an increase 
in the accuracy of detection of each feature quantity 
data item. 

[0083] A description will now be given of the opera- 
tions of the edge voting section 14 and the linear-line is 
extracting section 15, which are executed when a 
method for performing projection on an image plane in 
angular directions is employed instead of Hough trans- 
form. 

[0084] In the edge voting section 14, the vertical 20 
edge emphasized image obtained in the edge empha- 
sizing section 1 1 is subjected to binarization using an 
appropriate threshold value, thereby extracting high- 
value pixels which typically appear at a fold or a wrinkle. 
At this time, the ink-printed portion is extracted together 25 
with noise. 

[0085] The flowchart of FIG. 15 illustrates the 
processing performed by the edge voting section 1 4 and 
the linear-line extracting section 15 after the extraction 
of pixels. In this case, first, the edge voting section 14 30 
executes processes at steps S31 - S34. More specifi- 
cally, to vary the angle to the center line SL2 in units of 
AO from -6c - +6c, -Be is set as the initial value of 6 
(S31). Then, the binarized pixels that contain noise and 
are arranged in a direction 9 are accumulated (S32). 35 
Subsequently, 6 is increased by A6 (S33), and it is 
determined whether or not 6 is greater than +6c (S34). 
Thus, one-dimensional accumulation data is obtained in 
each direction 6 by repeating the above processing with 
the value of 0 increased in units of A8 until 6 exceeds 40 
+6c. 

[0086] After that, the linear-line extracting section 
1 5 calculates the peak value of the obtained one-dimen- 
sional accumulation data in each direction of 9, to detect 
9m at which a maximum accumulation data peak is 45 
obtained (S35). Then, a linear line area of a predeter- 
mined width is determined in the direction of 8m (S36). 
thereby extracting only those pixels existing in the lin- 
ear-line area, which are extracted by binarization. 
Thereafter, the number of the extracted pixels is so 
counted by similar processing to that performed at the 
steps S24 and S25 of the Hough transform process 
(S3 7), and the average density of the extracted pixels 
obtained when the original image is input thereto is 
measured (S38). 55 
[0087] Referring then to the flowchart of FIG. 16, a 
description will be given of the entire procedure of deter- 
mining processing executed in the second embodiment. 



[0088] First, an IR image of the printed matter P2 is 
input by the IR image input section 10 (S41). and a par- 
ticular area including the printed area R2 is extracted 
(S42). Then, the edge emphasizing section 1 1 performs 
vertical edge emphasizing processing to create an edge 
emphasized image, in order to detect a vertical fold or 
wrinkle (S43). 

[0089] Subsequently, the edge voting section 14 
performs binarization on the vertical edge emphasized 
image, using an appropriate threshold value (S44), 
thereby extracting a linear-line area by the linear-line 
extracting section 1 5, and counting the number of high- 
value pixels that typically appear at the extracted linear 
fold and measuring the average density of the pixels 
(S45). The processing at the step S45 is executed using 
either Hough transform described referring to FIG. 14 or 
15, or projection processing on an image plane. After 
that, the determining section 13 determines the soil 
degree of the basis of each feature quantity data item 
(concerning the number and average density of 
extracted pixels) (S46), thereby outputting the soil 
degree determination result (S47). 
[0090] The structure of the soil degree determining 
apparatus of the second embodiment is similar to that of 
the first embodiment shown in FIG. 9. except that the 
contents of a program stored in the memory 32 are • 
changed to those illustrated in FIG. 16. 
[0091 ] A third embodiment of the invention will be 
described. 

[0092] In the above-described second embodiment, 
a fold of the printed area R2 of the printed matter P2 is 
extracted to determine the soil degree, if, in this case, a 
cutout space or a hole is formed in the fold as shown in 
FIG. 1 7, it is difficult to extract only the fold for the follow- 
ing reason: 

[0093] In the vertical edge emphasizing process 
using the edge emphasizing section 1 1 in the second 
embodiment emphasizing processing is executed not 
only on a point of change at which the brightness is 
lower than that of the other horizontal points, but also on 
a point of change at which the brightness is higher than 
that of the other horizontal points. In other words, in the 
image input operation using transmitted IR light, even a 
hole or a cutout space in a fold, in which the brightness 
is at high level, is emphasized in the same manner as 
the fold whose brightness is at low level. Accordingly, 
the fold cannot be discriminated from the hole or the 
cutout space by subjecting an edge emphasized image 
to binary processing using an appropriate threshold 
value. 

[0094] To solve this problem, the third embodiment 
uses the feature that any fold has a low brightness (high 
density) in an image input using transmitted IR light. In 
other words, an input image is subjected to horizontal 
maximum filtering processing instead of the edge 
emphasizing processing, so that only pixels contained 
in a change area, in which the brightness is higher than 
that of the other horizontal area, can be extracted. The 
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input image is subtracted from the resultant image of a 
maximum value, and binary processing is executing 
using an appropriate threshold value, to extract only a 
fold. Further, individual extraction of a hole or a cutout 
space enables individual calculation of feature quantity 5 
data items concerning a fold, a hole or a cutout space, 
thereby enhancing the reliability of soil degree determi- 
nation results. 

[0095] FIG. 1 8 schematically shows the structure of 
a soil degree determination apparatus, according to the 10 
third embodiment for determining soil degree of printed 
matter. The apparatus of the third embodiment differs 
from that of the second embodiment in the following 
points. An IR image input section 10 shown in FIG. 18 is 
similar to the IR image input section 10 of FIG. 13 is 
except that in the former, an image is input using only 
transmitted IR light as shown in FIG. 5A. Further, an 
edge voting section 14 and a linear-line extracting sec- 
tion 15 shown in FIG. 18 have the same structures as 
the edge voting section 14 and the linear-line extracting 20 
section 15 shown in FIG. 13. However, a determining 
section 13 in FIG. 18 differs from that of FIG. 13 in that 
in the former, feature quantity data concerning a hole 
and/or a cutout space is input Also in the third embodi- 
ment, a determination result similar to that obtained 25 
from humans can be output by newly setting a determi- 
nation reference based on each feature quantity data 
item, as described in the first embodiment. 
[0096] A maximum/minimum filter section 16. a dif- 
ference image generating section 1 7 and a hole/cutout- 30 
space extracting section 18 will be described. 
[0097] FIGS. 19A to 19D are views useful in 
explaining the operations of the maximum/minimum fil- 
ter section 16 and the difference image generating sec- 
tion 17. FIG. 19A shows a brightness distribution 35 
contained in data on an original image, and FIG. 19B 
shows the result of a maximum filtering operation per- 
formed on the (5 x 1) pixels contained in the original 
image data of FIG: 1 9A, which include a target pixel and 
its adjacent ones. The maximum filter replaces the 40 
value of the target pixel with the maximum pixel value of 
horizontal five pixels that include the target pixel and 
horizontal four pixels adjacent thereto. 
[0098] By the maximum filtering operation, in an 
edge area in which the brightness is low within a width 45 
of four pixels, the brightness is replaced with a higher 
brightness obtained from a pixel adjacent thereto, 
thereby eliminating the edge area. The maximum 
brightness of edge pixels having high brightnesses is 
maintained. so 
[0099] FIG. 19C shows the result of a minimum fil- 
tering operation executed on the operation result of FIG. 
19B. The minimum filter performs, on the result of the 
maximum filtering operation, an operation for replacing 
the value of the target pixel with the minimum pixel value 55 
of the horizontal (5x1) pixels that include the target 
pixel as a center pixel. As a result, edge areas A and B 
shown in FIG. 19A disappear in which the brightness is 



low within a width of four pixels, while an edge area C 
with a width of five pixels is maintained, as is shown in 
FIG. 19C. 

[0100] The difference image generating section 17 
calculates the difference between the maximum/mini- 
mum filtering operation result obtained by the maxi- 
mum/minimum f titer section 1 6, and image data input by 
the IR image input section 10. Specifically, a difference 
g(i.j) given by the following equation (4) can be 
obtained: 

g(ij) = min{max(f(i j))} - f(i,j) (4) 

where (i,j) represents the position of each pixel in the 
extracted area, f represents the input image, and min 
{max (f(i.j))} represents the maximum/minimum filtering 
operation. 

[0101] FIG. 19D shows the result of subtraction of 
the original image data of FIG. 19A from the minimum 
filtering operation result of FIG. 19C. As is evident from 
FIG. 19D. only the edge areas A and B in which the 
brightness is low within a width of four pixels are 
extracted. 

[0102] From the operation results of the maxi- 
mum/minimum filter section 16 and the difference image 
generating section 1 7. the value g(i,j) of an edge area in 
which the brightness is lower than that of the other hor- 
izontal area is g(i.j) > 0. while the value g(i j) of an edge 
area in which the brightness is higher than that of the 
other horizontal area is g(i.j) = 0. 
[0103] The hole/cutout-space extracting section 18 
will be described. In the case of image input using trans- 
mitted IR light, light emitted from the light source directly 
reaches the CCD image sensor through a hole or a cut- 
out space. Therefore, the brightness of the hole or the 
cutout space is higher than the brightness of the non- 
printed area of printed matter, which is relatively high. 
For example, in a case where an 8-bit A/D converter is 
used and the printed area of printed matter has a bright- 
ness of 128 (= 80 h). a hole or a cutout space formed 
therein has a saturated brightness of 255 (= FFh). 
Accordingly, pixels corresponding to a hole or a cutout 
space can easily be extracted by detecting pixels of 
"255" in an area extracted from an image which has 
been input using transmitted IR light. The number of 
extracted pixels corresponding to a hole or a cutout 
space is counted and output. 

[01 04] Referring now to the flowchart of FIG. 20. the 
entire procedure of the determining process employed 
in the third embodiment will be described. 
[01 05] First the IR image input section 1 0 inputs an 
IR image of the printed matter P2 (S51). thereby 
extracting a particular area including the printed area 
R2 (S52). Subsequently, the maximum/minimum filter 
section 16 executes horizontal maximum/minimum fil- 
tering processing to create maximum/minimum filter 
image (S53). Then, the difference image generating 
section 1 7 creates a difference image by subtracting the 
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input image data from the maximum/minimum fitter 
image data (S54). 

[0106] After that, the edge voting section 14 per- 
forms binary processing on the difference image, using 
an appropriate threshold value (S55). and the edge vot- s 
ing section 14 and the linear-line extracting section 15 
extract a linear-line area as a fold. Thereafter, the linear- 
line extracting section 15 counts the number of high- 
value pixels which typically appear at the extracted fold, 
and measures the average density of the extracted pix- w 
els obtained when the original image is input thereto 
(S56). 

[0107] After that, the hole/cutout-space extracting 
section 18 measures the number of pixels correspond- 
ing to a hole or a cutout space (S57), and the determin- is 
ing section 1 3 determines the soil degree of the basis of 
each measured feature quantity data item (the number 
and the average density of extracted pixels, and the 
number of pixels corresponding to a hole or a cutout 
space) (S58), thereby outputting the soil degree deter- 20 
mination result (S59). 

[0108] The soil degree determining apparatus of 
the third embodiment has the same structure as the first 
embodiment described referring to FIG. 9, except that in 
the former, the contents stored in the memory 32 are 25 
changed to those illustrated in the flowchart of FIG. 20. 
[0109] A fourth embodiment of the invention will be 
described. 

[01 1 0] In the above-described second embodiment, 
a fold can be extracted even when the printed area R2 30 
of the printed matter P2 is printed with ink containing 
carbon, as well as chromatic color ink. 
[0111] However, rf in the second embodiment, the 
vertical lines of letters are superposed upon the center 
line SL2, the accuracy of extraction of a fold that will 35 
easily occur on and near the center line SL2 will reduce. 
[0112] FIG. 21 A shows an example of a soil, which 
reduces the accuracy of determination of a soil in the 
second embodiment. Printed matter P3 shown in FIG. 
21 A consists of a printed area R3 and a non-printed 40 
area Q3. The printed area R3 includes a center line SL3 
that divides, into left and right equal portions, the printed 
matter P3 that has a longer horizontal side than a verti- 
cal side, and also includes a printed pattern and letter 
strings STR1 and STR2 printed in black ink. The reflect- 45 
ance of the black ink is substantially equal to that of a 
fold. Assume that a fold or a wrinkle will easily occur 
near the center line SL3 as in the case of the center line 
SL1 of the printed matter P1 . 

[0113] As described in the second embodiment, a so 
letter pattern included in a pattern in the printed area R3 
will appear as noise when the pattern is subjected to 
binarization. Further, in the case of the printed matter 
P3, each vertical line of letters "NT and "IT contained in 
the letter strings STR1 and STR2 is aligned with the ss 
center line SL3. Accordingly, when the pattern in the 
printed area R3 has been binarized, the vertical lines of 
the letters are extracted as a fold as shown in FIG. 21 B. 



Thus, even rf there is no fold, it may erroneously be 
determined, because of the vertical line of each letter, 
that a linear line (a fold) exists. 

[0114] To avoid such erroneous determination and 
hence enhance the reliability of the linear line extraction 
processing, in the fourth embodiment, a letter-string 
area is excluded from an area to be processed as 
shown in FIG. 21 C where the letter-string area is prede- 
termined in the printed area R3 of the printed matter P3. 
[0115] FIG. 22 schematically shows a soil degree 
determining apparatus for printed matter according to 
the fourth embodiment The soil degree determining 
apparatus of the fourth embodiment has the same 
structure as that of the second embodiment, except that 
the former additionally includes a mask area setting 
section 19. 

[0116] The mask area setting section 19 will be 
described. In the case of a to-be-processed area 
extracted by the IR image input section 10, it is possible 
that a letter-string area cannot accurately be masked 
because of inclination or displacement of printed matter 
during its transfer. To accurately position a to-be- 
masked area so as to exclude a letter string from a to- 
be-processed target, it is necessary to accurately detect 
the position of the printed matter P3 when its image is - 
input, and to set a to-be-masked area on the basis of 
the detection result. This processing is executed in 
accordance with the f bwchart of FIG. 23. 
[0117] First, the entire portion of an input image of 
the printed matter P3, which is input so that the entire 
printed matter P3 will always be included, is subjected 
to binarization processing (S61). At a step S62, the 
positions of two points on each side of the printed mat- 
ter P3 are detected, in order to detect an inclination of 
the printed matter, by sequentially detecting horizontal 
and vertical pixel-value-changed points beginning from* 
each end point of the resultant binary image. Then, the 
positions of the four linear lines of the printed matter P3 
are determined, thereby calculating intersections 
between the four linear lines, and determining the posi- 
tion of the printed matter. 

[0118] At a step S63, the position of any to-be- 
masked area in the input image is calculated on the 
basis of the position and the inclination calculated at the 
step S62, and also on the basis of prestored position 
information on the to-be-masked area(s) of the printed 
matter P3. 

[0119] Referring to the flowchart of FIG. 24, the 
entire procedure of determining processing performed 
in the fourth embodiment will be described. 
[01 20] First, the IR image input section 1 0 inputs an 
IR image of the printed matter P3 (S71), thereby 
extracting a particular area including the printed area 
R3 and setting a to-be-masked area by the mask area 
setting section 19 as illustrated in FIG. 23 (S72). Subse- 
quently, the edge emphasizing section 1 1 executes ver- 
tical emphasizing processing to create a vertical-edge- 
emphasized image (S73). 
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[0121] After that, the edge voting section 14 exe- 
cutes binarization of the vertical-edge-emphasized 
image, using an appropriate threshold value (S74). At 
the next step S75, the edge voting section 14 and the 
linear line extracting section 15 detect a linear-line area, 
and obtain the number of high-value pixels that typically 
appear at a fold in the extracted linear-line area, and 
also the average density of these pixels, which is 
obtained when the original image is input thereto. The 
determining section 13 determines the soil degree of 
the basis of the measured feature quantity data (the 
number and the average density of the extracted pixels 
obtained when the original image is input) (S76), 
thereby outputting the soil degree determination result 
(S77). 

[0122] The soil degree determining apparatus of 
the fourth embodiment has the same structure as the 
first embodiment described referring to FIG. 9. except 
that in the former, the contents stored in the memory 32 
are changed to those illustrated in the flowchart of FIG. 

24. 

[0123]^ A fifth embodiment of the invention will be 
described: 

[0124] FIG. 25 shows an example of printed matter 
that has a soil to be checked in the fifth embodiment 
Printed matter P4 shown in FIG. 25 has a tear at an 
edge thereof. Where a tear occurs in the flat printed 
matter P4, one of two areas divided by the tear gener- 
ally deforms at an angle (upward or downward) with 
respect to the flat printed surface as shown in FIGS. 
26A and 26B. In the case of inputting an image by using 
usual transmitted light, a light source is located perpen- 
dicular to the printed surface, while a CCD image sen- 
sor is located opposite to the light source, with the 
printed surface interposed therebetween. 
[0125] If an image having a tear is input in the 
above structure, it is possible, unlike a hole or a cutout 
space/that light from the light source will not enter the 
CCD image sensor. Specifically, like a fold, a tear is 
detected as a change in brightness from a bright portion 
to a dark portion, depending upon the angle, to the 
printed surface, of a line formed by connecting the light 
source and the CCD image sensor. Further, even if light 
from the light source will directly enter the CCD image 
sensor when the printed surface and the tear form a cer- 
tain angle, it cannot directly enter the CCD sensor if the 
tear is formed as shown in FIG. 26A or 26B. 
[0126] To distinguish a tear from a fold or a wrinkle 
in a reliable manner, at least two image input means 
must be used. 

[0127] FIG. 27 schematically illustrates the struc- 
ture of a soil degree determining apparatus for printed 
matter according to the fifth embodiment The soil 
degree determining apparatus of the fifth embodiment 
has two transmitted-image input sections 20a and 20b 
in a different direction from its transfer direction. The 
sections 20a and 20b input respective image data items 
obtained using transmitted light and corresponding to 



the printed matter P4 that includes a soil having 
occurred near the center line SL4. thereby extracting a 
particular area contained in the input image data items. 

[0128] Tear extracting sections 21a and 21b extract 

5 a torn area from the image data contained in the partic- 
ular area extracted by the transmitted-image input sec- 
tions 20a and 20b, and measure the number of pixels 
included in the torn area. The determining section 13 
determines the soil degree of the printed matter P4 on 

10 the basis of the number of pixels measured by the tear 
extracting sections 21a and 21b. 
[0129] The transmitted- image input sections 20a 
and 20b will be described. Each of these sections 20a 
and 20b has the same structure as the IR image input 

15 section 10 (with the structure shown in FIG. 5A) except 
that the former does not have the IR filter 3. 
[0130] FIGS. 28A and 28B show optical arrange- 
ments of the transmitted -image input sections 20a and 
20b. To detect vertically displaced tears as shown in 

20 FIGS. 26A and 26B. it is necessary to arrange, as 
shown in FIG. 28A or 28B two input sections having an 
optical angle of ±6 (0 < 8 < 90° ) with respect to the 
printed surface. The closer the value of e to "0", the eas- 
ier the detection of a tear and the higher the detection 

25 accuracy of the tear. This is because the closer to M 0 M , 
the greater the physical displacement of the tear. 
[0131] Specifically, in the structure shown in FIG. 
28A, a first light source 2a is located above the printed 
matter P4, and a first lens 4a and a first CCD image 

30 sensor 5a are located below the printed matter P4, 
opposed to the first light source 2a. In addition, a sec- 
ond light source 2b is located below the printed matter 
P4, and a second lens 4b and a second CCD image 
sensor 5b are located above the printed matter P4, 

35 opposed to the second light source 2b. 

[0132] In the structure shown in FIG. 28B, the first 
and second light sources 2a and 2b are located above 
the printed matter P4, while the first and second lenses 
4a and 4b and the first and second CCD image sensors 

40 5a and 5b are located below the printed matter P4, 
opposed to the light sources 2a and 2b, respectively. 
[01 33] The tear extracting sections 21a and 2 1 b will 
be described. Since these sections have the same 
structure, a description will be given only of the tear 

45 extracting section 21a The tear extracting section 21a 
executes similar processing on image data contained in 
the particular area extracted by the transmitted-image 
input section 20a, to the processing executed by the 
hole/cutout-space extracting section 18 shown in FIG. 

so 18. 

[01 34] Specifically, where an 8-bit A/D converter, for 
example, is used and the paper sheet has a brightness 
of 128 (= 80h), if the transmitted-image input section 
20a receives direct light through a tear as through a fold, 
55 it outputs a saturated value of 255 (FFh). Therefore, if a 
pixel that assumes a value of "255** is detected in the 
particular area extracted by the transmitted-image input 
section 20a, a tear can be easily detected. The tear 
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extracting section 21a counts and outputs the number of 
thus-extracted pixels corresponding to a tear. 

[0135] The determining section 13 will be 
described. The determining section 13 sums the 
counted numbers of pixels corresponding tears to deter- s 
mine the soil degree of the printed matter P4. A refer- 
ence value used in the determination is similar to that 
used in the first embodiment. 

[01 36] Referring now to the flowchart of FIG. 29, the 
entire procedure of the determining process employed 10 
in the fifth embodiment will be described. 
[0137] First, the transmitted -image input sections 
20a and 20b input images of the printed matter P4 (S81 , 
S82), thereby extracting particular areas (S83, S84). 
Subsequently, the tear extracting sections 21a and 21b is 
detect, from the input images, pixels that have 
extremely high brightnesses, thereby counting the 
number of the detected pixels (S85, S86). Subse- 
quently, the determining section 13 determines the soil 
degree of the basis of the detected pixels (S87), and 20 
outputs the determination result (S88). 
[0138] The structure of the soil degree determining 
apparatus of the fifth embodiment is realized by adding 
another image input section to the structure of the f irst 
embodiment shown in FIG. 9. In other words, a pair of 2s 
transmitted-image input sections 20a and 20b and a 
pair of image memory control sections 34a and 34b are 
employed as shown in FIG. 30. However, it is not always 
necessary to employ an IR filter. Moreover, the contents 
stored in the memory 32 are changed to those illus- 30 
trated in the flowchart of FIG. 29. 
[0139] A sixth embodiment of the invention will be 
described. 

[0140] Although the fifth embodiment uses the two 
transmitted-image input sections 20a and 20b for 35 
extracting tears of printed matter, the sixth embodiment 
described below and having a different structure from 
the fifth embodiment can also extract a tear without 
erroneously recognizing it to be a fold. 
[0141] As described in the fifth embodiment, a tear 40 
may be erroneously determined to be a fold or a wrinkle 
that is formed at en edge of printed matter, if an image 
of a torn portion of the printed matter is input by only 
one image input system using transmitted light To 
determine a tear by only one image input system using 45 
transmitted light, it is necessary to cause the CCD 
image sensor to directly receive, within its field of view, 
light emitted from the light source and having passed 
through a gap between two areas divided by a tear. 
[0142] In other words, it is necessary to transfer so 
printed matter so that a sufficient distance will be 
defined between two portions of the matter divided by a 
tear, on a plane perpendicular to a line formed by con- 
necting the light source and the CCD image sensor, i.e. 
so that a clear gap is defined between the two portions 55 
divided by the tear. To this end, the printed matter is 
bent using its elasticity and a force is applied to each of 
the two portions to widen the gap therebetween, as is 



shown in FIG. 31. 

[01 43] FIG. 32 schematically shows the structure of 
a soil degree determining apparatus for printed matter 
according to the sixth embodiment. FIG. 33A is a sche- 
matic top view showing a printed matter transfer system 
employed in the apparatus of FIG. 32, while FIG. 33B is 
a perspective view of the printed matter transfer system 
of FIG. 32. 

[0144] In FIG. 32, after transferred in a direction 
indicated by the arrow, the printed matter P4 is further 
moved at a constant speed by transfer rollers 41 and 42 
to a disk 43. where the matter P4 is pushed upward. 
While the printed matter P4 is urged against a transpar- 
ent guide plate 44, the printed matter P4 is directed 
down to the lower right in FIG. 32, and the printed mat- 
ter P4 is pulled by transfer rollers 45 and 46. 
[0145] In the above-described structure, a light 
source 2 applies light onto the printed matter P4 from 
above the center of the disk 43, with the transparent 
guide plate 44 interposed therebetween, and the CCD 
image sensor 5 receives light transmitted through the 
printed matter P4. An image signal obtained by the CCD 
image sensor 5 using transmitted light is input to a 
transmitted-image input section 20. 
[0146] The transmitted-image input section 20 is., 
similar to the transmitted-image input section 20a or 
20b employed in the fifth embodiment, except that the 
former does not include optical system units such as the 
light source 2, the lens 4 and the CCD image sensor 5. 
[0147] The transmitted-image input section 20 con- 
verts, into digital data, the input transmitted-image data 
indicative of the printed matter P4, using an A/D con- 
verter circuit, thereby storing the digital data in an image 
memory and extracting a particular area therefrom. A 
tear extracting section 21 extracts a tear and counts the 
number of pixels corresponding to the tear. A determin- 
ing section 13 determines the soil degree of the printed 
matter P4 on the basis of the counted number of the pix- 
els. 

[0148] The tear extracting section 21 and the deter- 
mining section 13 have the same structures as the tear 
extracting section 21a and the determining section 13 
employed in the fifth embodiment shown in FIG. 27. 
[0149] A description will now be given of the state of 
the printed matter P4 obtained when an image thereof 
is input, when the center line SL4 of the printed matter 
P4, at which soiling will easily occur, has reached an 
uppermost portion of the disk 43, the horizontal ends of 
the printed matter P4 are held between the transfer roll- 
ers 41 and 42 and between the transfer rollers 45 and 
46, respectively. 

[0150] Accordingly, that portion of the printed mat- 
ter P4, which is positioned on the uppermost portion of 
the disk 43. is warped. Therefore, if there is a tear on the 
center line SL4 at which soiling will easily occur, the 
same state occurs as that mentioned referring to FIG. 
31. As a result, the two areas, divided by the tear and 
located on a plane perpendicular to the line formed by 
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connecting the light source 2 to the CCD image sensor 
5, separate from each other, which enables extraction of 
the tear as in the fifth embodiment 

[0151] Referring to the flowchart of FIG. 34, the 
entire procedure of determining processing executed in 
the sixth embodiment will be described. 
[01 52] First, the transmitted-image input section 20 
inputs an image of the printed matter P4 (S91), thereby 
extracting a particular area (S92). Subsequently, the 
tear extracting section 21 extracts pixels of extremely 
high brightnesses from the input image, and counts the 
number of the extracted pixels (S93). After that the 
determining section 13 determines the soil degree of 
the basis of the counted number of the pixels (S94), and 
outputs the determination result (S95). 
[0153] The soil degree determining apparatus of 
the sixth embodiment has the same structure as the first 
embodiment except that the former does not include the 
I R image input section 1 0 (having the structure shown in 
FIG. 5A) using transmitted light, and the IR filter 3. 
[0154] The gist of the present invention does not 
change even if similar soil called, tor example, "a bend** 
or "a curve" is detected instead of "a fold", "a tear", "a 
hole" or "a cutout space" detected in the above embod- 
iments. 

[01 55] Moreover, although an area of printed matter 
transferred in a direction parallel to its length, which 
includes the vertical center line and its vicinity, is proc- 
essed in the above-described embodiments, the inven- 
tion is not limited to this. For exanple. the invention can 
also process an area of printed matter transferred in a 
direction parallel to its width, which includes the hori- 
zontal center line and its vicinity, or areas of printed mat- 
ter divided into three portions, which include two 
horizontal lines and their vicinities. 
[0156] In addition, the area from which a fold or a 
tear can be detected is not limited to an area within 
printed matter as shown in FIG. 7. Any area can be 
detected only if it is located within a certain distance 
from the center line SL1 in FIG. 1 A. 
[0157] As described above in detail, the present 
invention can provide a soil degree determining appara- 
tus that can determine, as humans do, a fold of a printed 
area of printed matter, unlike the conventional appara- 
tuses. 

[01 58] The invention can also provide a soil degree 
determining apparatus capable of discriminating 
between a fold and a tear of printed matter, which can- 
not be distinguished in the prior art. 

Claims 

1 . A soil degree determining apparatus for determin- 
ing soiling on printed matter, characterized by com- 
prising: 

image input means (10) for inputting an IR 
image of printed matter to be subjected to soil- 



ing determination, using IR light having a near- 
infrared wavelength; 

image extracting means (S2) for extracting 
image data in a particular area including a 
5 printed area, from the IR image input by the 

image input means; 

changed-section extracting means (S5, S6) for 
extracting, on the basis of the image data in the 
particular area extracted by the image extract- 
io ing means, a non-reversible changed section 

caused when the printed matter is folded, 
thereby providing data concerning the changed 
section; 

feature quantity extracting means (S7 - S9) for 
is extracting a feature quantity indicative of a 

degree of non-reversible change in the particu- 
lar area, on the basis of the data concerning 
the changed section and provided by the 
changed-section extracting means; and 
20 determining means (1 3) for estimating the fea- 

ture quantity extracted by the feature quantity 
extracting means, thereby determining a soil 
degree of the printed matter. 

25 2. An apparatus according to claim 1 , characterized in 
that the image input means (10) has an IR filter (3) 
for filtering wavelength components other than the 
near-infrared wavelength. 

30 3. An apparatus according to claim 1 , characterized in 
that the image input means (10) inputs the IR image 
of the printed matter, using one or both of light 
transmitted through the printed matter and light 
reflected from the printed matter. 

35 

4. An apparatus according to claim 1 , 

characterized in that the feature quantity extracting 
means includes at least one of extracted -pixel 
counting means for counting pixels corresponding 

40 to the data concerning the changed section 
extracted by the changed-section extracting 
means; average density measuring means for 
measuring that average density of the pixels corre- 
sponding to the changed section, which is obtained 

45 when the IR image is input by the image input 
means; and means for calculating a variance, in the 
particular area, of the pixels corresponding to the 
extracted changed section. 

so 5. An apparatus according to claim 1 , 

characterized by further comprising linear-line 
determining means for determining a linear-line 
area in the particular area on the basis of the data 
concerning the changed section provided by the 

55 changed-section extracting means, 

and characterized in that the feature quantity 
extracting means includes extract ed-pixel counting 
means for counting pixels in the linear-line area 
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determined by the linear-line determining means, 
and average density measuring means (S45) for 
measuring an average density of the pixels in the 
linear-line area, which is obtained when the IR 
image is input by the image input means. 5 

6. An apparatus according to claim 1 , characterized in 
that the changed-section extracting means has 
means (19) for masking a predetermined area in 
the particular area, thereby extracting a non-revers- 10 
ible changed section included in the particular area 
except for the predetermined area, and providing 
data concerning the changed section. 

7. An apparatus according to claim 1 , characterized in is 
that the image input means has first and second 
image input means (20a, 20b) using transmitted 
light, and the first and second image input means 
each have tear extracting means for extracting pix- 
els indicative of a tear which is formed at an edge 20 
portion of the printed matter, and providing a 
number of the extracted pixels as the feature quan- 
tity. 

8. An apparatus according to claim 1 , characterized in 25 
that the changed-section extracting means includes 
image emphasizing means (S3, S4) for emphasiz- 
ing a non-reversible change in the particular area 
caused when the printed matter is folded, and pro- 
viding emphasized image data. 30 

9. An apparatus according to claim 8, characterized in 
that the image emphasizing means emphasizes the 
non-reversible change in the particular area, using 

a pixel weight matrix. 35 

10. An apparatus according to claim 8, characterized in 
that the image emphasizing means emphasizes the 
non-reversible change in the particular area, using 

a maximum/minimum filter. 40 



45 



50 



55 



15 



BNSDOCID: <EP 



1011079A1_L> 



BNS page 15 



EP1 011 079 A1 




LU 
O 



o 

y 

Li. 



PAPER SHEET 



400 



WAVELENGTH [nm] 

FIG. 2A 



800 



8 



3 



CHROMATIC 
COLOR INK 



400 800 
WAVELENGTH [nm] 

FIG. 2B 



LU 
O 



PAPER SHEET 



-''CHROMATIC 

COLOR i nk x ;>' 

x FOLD -WRINKLE (BLACK) 



400 800 
WAVELENGTH [nm] 

FIG. 2C 



16 



079A1 I _> 



EP1 011 079 A1 



TO SENSOR 

L I GHT-RECE IVING 
A 



LIGHT FROM 
LIGHT SOURCE ! 



\\\ 

I 



DARK 
PORT I ON 




i 

i i 




BRIGHT 
PORT I ON 



FOLDED 
PORT I ON 



FIG. 3A 



TO SENSOR 
LIGHT-RECEIVING 

A 



LIGHT FROM 
LIGHT SOURCE 



\\\ 

4r 



BR I GHT 
PORT I ON 



FOLDED 
PORTION 



DARK 
PORT I ON 



FIG. 3B 



PI 


10 




11 




s 


J 




IR 




EDGE 


PR 1 NTED /. 


IMAGE 




EMPHA- 


MARKER J 


INPUT 




SIZING 




SECT 1 ON 




SECT I ON 



12 

1 



FOLD- 
WRINKLE 
EX- 
TRACT I NG 
SECT I ON 



13 

1 



DETER- 
MINING 
SECT I ON 



BLEM I SH 
DEGREE 
DETER- 
MINATION 
RESULT 



FIG. 4 



pi 



V 



CCD 




IMAGE 


^5 


SENSOR 








I ■ 1 


r-3 




1 




P1 

1 





CCD 


5- — 


IMAGE 




SENSOR 


4 ■ 


A 




1 





FIG. 5A 



FIG.5B 



17 



1011079A1J_> 



EP1 011 079 A1 



POSITION SENSOR 



TRANSFER CLOCK 

CCD IMAGE LINE 

SENSOR TRANSFER 

DIRECTIONAL 

EFFECTIVE PERIOD 
TWO-DIMENSIONAL 
CCD SENSOR SHUTTER 
EFFECTIVE PERIOD 



FIG.6 



r PRINTED MATTER 
! SHADED PERIOD 




FIRST DELAY PERIOD 



I 



—? , 

SECOND DELAY PERIOD 

t 

n 





FIG.7A 



FIG. 7B 



-1 


0 


1 


-1 


0 


1 


-1 


0 


1 



FIG.8A 



-1 


-1 


-1 


0 


0 


0 


1 


1 


1 



FIG.8B 



18 



BNSDOCID: <EP 



1011079A1_I_> 



BNS page 18 



EP1011079A1 




32 

1 



MEMORY 



33 

_L 

DISPLAY 
SECT I ON 



36 

L 



10 

I 

\ 





5 



CCD 

IMAGE 

SENSOR 



r 

-"1 


39 




40 1 


1 3 f 




, L 




r— y ~A 








CONTROL 




IMAGE 


* 


IMAGE DATA 




CIRCUIT 




MEMORY 




l/F CIRCUIT 



38 



I 



A/D 

CONVERTING 
CIRCUIT 



T 



IMAGE DATA 



1 



TIMING 

CONTROL 

CIRCUIT 

r 

37 




1 



FIG.9 



BNSDOCIO: <EP 



1011079A1J_> 



19 



BNS page 19 



EP 1 011 079 A1 




START 



I 



INPUT IR IMAGE 




S1 



CREATE VERTICALLY 
EMPHASIZED IMAGE 



CREATE 
BINARY IMAGE 



I 



COUNT EXTRACTED 
PIXELS 



1 



MEASURE AVERAGE 
BRIGHTNESS OF 
EXTRACTED PIXELS 



CALCULATE VARIANCE 
OF COORDINATE 
VALUES (POSITIONS) 



EXTRACT TO-BE-PROCESSED AREA p- S2 



1 



CREATE HORIZONTALLY 
EMPHASIZED IMAGE 



I 



CREATE 
BINARY IMAGE 



I 



COUNT EXTRACTED 
PIXELS 



I 



MEASURE AVERAGE 
BRIGHTNESS OF 
EXTRACTED PIXELS 



DETERMINATION PROCESSING h-S12 

7 

OUTPUT DETERMINATION RESULT I — S13 

FIG . 10 C^D 



20 



EP1 011 079 A1 



R2 
Q2 





7 ///a 










1 





SL2 




FIG.11A 



FIG. 1 IB 



LU 



o 

LU 

U_ 
LU 




CHROMATIC COLOR INK 



CHROMATIC COLOR INK+ 



BLACK INK 
BLACK INK 



400 800 
WAVELENGTH [ran] 

FIG. 12 



P2 

A 



PR I NTED 
MATTER 




10 

i 



IR 




IMAGE 




INPUT 




SECT 1 ON 





11 



14 

I 



EDGE 




EDGE 




L 1 NEAR-L 1 NE 


EMPHAS 1 Z 1 NG 




V0T 1 NG 




EXTRACT 1 NG 


SECT 1 ON 




SECT 1 ON 




SECT 1 ON 



15 

I 



FIG. 13 



13 



DETERM INING 
SECT I ON 



BLEM I SH 
DEGREE 

DETERM I NAT I ON 
RESULT 



21 



BNSOOCID: <EP 



1011079A1J_> 



BNS page 21 



EP1 011 079 A1 



START 



I 



PERFORM HOUGH TRANSFORM 

i 

PERFORM BINARIZATION 
ON HOUGH PLANE 



i 



EXTRACT EDGE PIXELS 
ON LINE AR LINE 

1 



c 



END 



S21 




S23 



COUNT EXTRACTED PIXELS 

i 




S24 



MEASURE AVERAGE BRIGHTNESS 
OF EXTRACT ED PIXE LS 




S25 



FIG. 14 



22 



BNSDOCID: <EP 1011079A1_L> 



EP 1 01 1 079 A1 



c 



START 



T 



0 — -0C 



3 



S31 



ACCUMULATE 6 -DIRECTIONAL 
EDGE PIXELS 



I 



S32 



0+-6 + A e 



S33 




S34 



S35 



CALCULATE dm AT WHICH 
MAXIMUM PEAK OF ACCUMULATED 
DATA IS OB TA I NED 

DETERMINE LINEAR-LINE AREA | — S36 



I 



COUNT EXTRACTED PIXELS 



S37 



I 



MEASURE AVERAGE BRIGHTNESS S38 
OF EXTRACTED PIXELS 



c 



I 



END 



FIG. 15 



23 

BNSDOCID: <EP 1011079A1_I_> 



EP1 011 079 A1 



Q START ^ 
I 

INPUT IR IMAGE p S41 

I 



EXTRACT TO-BE-PROCESSED AREA h- S42 

r — 



CREATE LENGTHWISE 
EMPHASIZED IMAGE 

T 



S43 



CREATE BINARY IMAGE [ — S44 

I 



COUNT EXTRACTED PIXELS 
CORRESPONDING TO LINEAR LINE 
AND MEASURE AVERAGE BRIGHTNESS 
OF EXTRACTED PIXELS 



S45 



1 



PERFORM DETERMINING PROCESSING h~S46 

I 

OUTPUT DETERMINATION RESULT |^- S47 



( END ) 

FIG. 16 



BNSDOCID: <EP 



1011079A1_I_> 



24 



BNS page 24 



EP1 011 079 A1 



P3 



i 



PRINTED 
MATTER 




10 

A 



TRANSMITTED 
IR IMAGE 
INPUT 
SECT I ON 



CUTOUT SPACE 




HOLE 



SL2 



FIG. 17 



16 

2 



MAXIMUM/ 

MINIMUM 

FILTER 



17 

L 



D I FFERENCE 
IMAGE 

GENERAT I NG 
SECTION 



HOLE/CUTOUT- 
SPACE 
EXTRACTING 
SECT I ON 



18 



14 

A 



EDGE 
VOT I NG 
SECTION 



15 
A 



LINEAR-LINE 
EXTRACT I NG 
SECT I ON 



13 

A 



DETER- 
MINING 
SECT I ON 



BLEM I SH 
DEGREE 
DETER- 
MINATION 
RESULT 



FIG- 18 



25 



BNSDOCID: <EP 



1011079A1J_> 



BNS page 25 



EP1 011 079 A1 



FIG.19A 




FI6.19B 



FIG.19C 



I I I I I I I I I I I I I I I I 1 



B 



I I l I 

C 



I I I 









JJhh 













FIG. 19D 



i — i p— n 

^■^■^^ i i I i i !■ i i i i 1 — till 



titi 



BNSDOCID: <EP. 



1011079A1_I_> 



26 



BNS page 26 



EP1 011 079 A1 



c 



START 




I 



TRANSM 



I 



S51 



EXTRACT TO-BE-PROCESSED AREA 



I 



S52 



CREATE IMAGE USING 
MAXIMUM/MINIMUM FILTER 



1 



S53 



CREATE DIFFERENCE IMAGE BY 
SUBTRACTING INPUT IMAGE FROM 
MAXIM UM/MINIMUM FILTER IMAGE 

— — i 



S54 



CREATE BIN 

~~ — 7 



COUNT EXTRACTED PIXELS 
CORRESPONDING TO LINEAR LINE 
AND MEASURE AVERAGE BRIGHTNESS 
OF EXTRACTED PIXELS 



S56 



COUNT EXTRACTED 
CORRESPONDING Tl 
OR CUTOUT SPACE 



I 



S57 



PROCESS 



I 




S58 



DETERMII 



S59 



c 



END 



3 



FIG. 20 



27 



BNSDOCIO: <EP 



1011079A1_I_> 



BNS page 27 



EP1 011 079 A1 



STR1 




STR2 



W » • 
• • " 


• • 

• • 


w • 


^^^fc^^ • • • 

::yn 


• • 

■ • • 

• • 

• • a 

• * 


1 • 

• 


.tost 

m m m 


IBU 

• * « 

• • * 


* 

* 

• 

* 



SL3 Q3 



FIG.21A 



FIG.21B 



P3 



R3 







a • • < 




• 
* 


■ • 


• • ■ a 

» . a < 

• a a a 


m 






P.: 









7 



TO-BE-MASKED AREA 



SL3 



FIG.21C 



28 



BNSDOCID: <EP 1011079A1J_> 



EP 1 01 1 079 A1 



P3 

_L 

'PR I NTED 
MATTER 




10 


IR 




IMAGE 




INPUT 




SECTION 





14 

_1 



EDGE 
VOT I NG 
SECTION 



19 

I 

TO-BE-MASKED 
AREA SETTING 
SECTION 



11 

\ 

EDGE 

EMPHASIZING 
SECT I ON 



1*5 



LINEAR-LINE 
EXTRACTING 
SECT I ON 



13 



DETERMINING 
SECT I ON 



BLEM I SH 
DEGREE 

DETERMINATION 
RESULT 



FIG. 22 



c 



START 



I 



PERFORM BINARIZATION 

I 



DETECT INCLINATION AND 
POSITION OF INPUT IMAGE 
OF PRINTED MATTER 



I 



SET TO-BE-MASKED AREA 



c 



I 



END 



S61 



S62 



S63 



FIG. 23 



29 



BNSDOCID: <EP 



1011079A1_I_> 



BNS page 29 



EP1 011 079 A1 




START 



I 




INPUT IR IMAGE 



i 



DETECT TO-BE-PROCESSED AREA 
AND TO-BE-MASKED AREA 



I 



CREATE VERTICALLY 
EMPHASIZED IMAGE 



i 



CREATE BINARY IMAGE 




S71 



S72 




S73 



S74 



COUNT EXTRACTED PIXELS 
CORRESPONDING TO LINEAR LINE 
AND MEASURE AVERAGE BRIGHTNESS 
OF EXTRACTED PIXELS 



i 



S75 



1 



PERFORM DETERMINING PROCESSING KS76 



OUTPUT DETERMINATION RESULT 

T 




S77 



c 



END 



J 



FIG. 24 



BNSDOCID: <EP 



1011079A1_I_> 



30 



BNS page 30 



EP1 011 079 A1 



P4 




TEAR 



FIG. 25 




P4 



TEAR 



TEAR 




F I G. 26A 



FIG.26B 



P4 



PRINTED 
MATTER 




20a 



21a 

I 





TRANSM 1 TTED 




TEAR 




IMAGE INPUT 




EXTRACT I NG 




SECTION 1 




SECTION 1 


20b 

\ 


21b 

/ 




TRANSM I TTED 




TEAR 




IMAGE INPUT 




EXTRACT I NG 




SECTION 2 




SECTION 2 



13 

Jl 



DETER- 
MINING 
SECT I ON 



BLEM I SH 
DEGREE 
DETER- 
MINATION 
RESULT 



FIG. 27 



31 



BNSDOCID: <EP 



1011079A1_L> 



» 



EP 1 011 079 A1 





FIG.28A 



FIG. 28B 



S81 



S83 



S85 



c 



START 



I 



INPUT TRANSMITTED 
IMAGE 1 



I 



1 



INPUT TRANSMITTED 
IMAGE 2 



S82 



EXTRACT 

TO-BE-PROCESSED 
AREA 



1 



T 



EXTRACT 

TO-BE-PROCESSED 
AREA 



-S84 



COUNT PIXELS 
CORRESPOND I NG 
TO TEAR 



i 



COUNT PIXELS 
CORRESPOND I NG 
TO TEAR 



T 



I 



PERFORM DETERMINING PROCESSING I — S87 



OUTPUT DETERMINATION RESULT 



FIG. 29 



c 



I 



S88 



END 



S86 



32 



BNSDOCID: <EP 



1011079A1_I_> 



BNS page 32 



v, 



EP 1 011 079 A1 



20a 




2a 




IMAGE DATA 
33 ♦ 35 

-L I I 



0 1 SPLAY 
SECTION 



i 



IMAGE 
DATA l/F 
CIRCUIT 



I 



36 

L 



i r 



34b 

l 



38a 




I 



CONTROL 




IMAGE 




CONTROL 




IMAGE 


CIRCUIT 1 




MEMORY 1 




CIRCUIT 2 




MEMORY 2 



A/D 

CONVERTING 
CIRCUIT 1 



39b 

I 



40b 



38b 



A/D 

CONVERT I NG 
CIRCUIT 2 



5a 



CCD 
IMAGE 
SENSOR 1 




2b 




5b 

S 



CCD 
IMAGE 
SENSOR 2 

— X — 



TIMING 

CONTROL 

CIRCUIT 



FIG. 30 



I 



37 



20b 



POSITION 
SENSOR 



1 



33 



BNSOOCIO: <EP. 



1011079A1J_> 



BNS page 33 



EP1 011 079 A1 




20 
\ 


21 

) 




13 
) 


BLEMISH 




TRANSMITTED 




TEAR 




DETER- 


DEGREE 




IMAGE INPUT 




EXTRACT 1 NG 




MINING 


■^DETER- 




SECT 1 ON 




SECT 1 ON 




SECT 1 ON 


MINATION 



RESULT 



FIG. 32 



EP1 011 079 A1 



41,42 




7 



45,46 



F I G. 33A 




F I G. 33B 



C START ) 

S91 



INPUT TRANSMITTED IR IMAGE 

J 




EXTRACT TO-BE-PROCESSED AREA H S92 



I 



COUNT PIXELS CORRESPONDING TO TEAR \ 



S93 



I 

S94 



PERFORM DETERMINING PROCESSING 

r — 

OUTPUT DETERMINATION RESULT l ~ S95 



( end ) 

FIG. 34 



r 



35 



EP1 011 079 A1 




European Patent 
Office 



EUROPEAN SEARCH REPORT 



Application Number 

EP 99 12 4928 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category 



Citation of document with indication, where appropriate. 
of relevant passages 



Relevant 
to claim 



CLASSIFICATION OF THE 
APPLICATION (lnt CI.7) 



US 4 710 963 A (LACEY PAUL D ET AL) 
1 December 1987 (1987-12-01) 

* the whole document * 

WO 96 36021 A (GIESECKE & DEVRIENT GMBH 
;H0LL N0RBERT (DE); H0LZNER FL0RIAN (DE);) 
14 November 1996 (1996-11-14) 

* page 5, line 4 - page 6, line 21 * 

* page 9, line 35 - page 17, line 8 * 

* figures 7-11 * 

US 5 055 834 A (CHIBA TERUHISA) 
8 October 1991 (1991-10-08) 

* column 2, line 60 - column 4, line 46 * 

US 4 650 319 A (KAULE WITTICH ET AL) 
17 March 1987 (1987-03-17) 



1-4 



G07D7/16 
G07D7/18 
G07D7/12 



1-3 



1 



TECHNICAL FIELDS 
SEARCHED (IntCl.T) 



G07D 



The present search report has been drawn up for all claims 



Ptoc* of ••••ch 

THE HAGUE 



Data of oornptotton of fh« Marcft 

13 March 2000 



Bocage, S 



si 

8 



i 

2 
2 

Hi 



CATEGORY OF CITED DOCUMENTS 

X : particularly relevant It taken alone 

Y : particularly relevant fl combined with another 

document of the same category 
A : technological background 
O : non-written disclosure 
P : intermediate document 



T : theory or principle underlying the invention 
E : earner patent document, but published on, or 

after the ruing date 
D : document cited in the application 
L : document cited for other reasons 



& : member of the same patent family, corresponding 



36 



BNSDOCID: <EP 



1011O79A1J_> 



BNS page 36 



' EP 1011 079 A1 



ANNEX TO THE EUROPEAN SEARCH REPORT 
ON EUROPEAN PATENT APPLICATION NO. 



EP 99 12 4928 



This annex lists the patent family members relating to the patent documents cited in the above-mentioned European search report 
The members are as contained in the European Patent Office EDP file on 

The European Patent Office is in no way liable lor these particulars which are merely given tor the purpose of information. 

13-03-2000 



Patent document 




Publication 




Patent family 




Publication 


cited in search report 




date 




members) 




date 


IK 4710061 


A 

n 




r d 
uB 




A 

A 


1 rt AO — 1 Qfl£ 

19 U.S— 1980 








CP 

tr 




A 

A 


1 A AO 1 QQC 








WU 


O0U19CJ 


A 

A 


ao 1 ADC 

Z/-03-1980 








ID 


0£bOU9b9 


T 

I 


If A A 1 AO 7 

10-04- 198/ 


W0 9636021 


A 


14-11-1996 


DE 


19517194 


A 


14-11-1996 








AU 


5816696 


A 


29-11-1996 








EP 


0824736 


A 


25-02-1998 








JP 


11510623 


T 


14-09-1999 


US 5055834 


A 


08-10-1991 


JP 


1897399 


C 


23-01-1995 








JP 


6014384 


B 


23-02-1994 








JP 


63255792 


A 


24-10-1988 








DE 


3812268 


A 


03-11-1988 








GB 


2204682 


A,B 


16-11-1988 


US 4650319 


A 


17-03-1987 


DE 


2932962 


A 


26-02-1981 








AT 


5552 


T 


15-12-1983 








EP 


0024053 


A 


18-02-1981 








JP 


1676536 


C 


26-06-1992 








JP 


3025837 


B 


09-04-1991 








JP 


56033530 


A 


04-04-1981 



1 

s _ ... 

in For more details about this annex : see Official Journal of the European Patent Office. No. 12/82 



37 



1011079A1_I_> 



